Perspectives on Learning Symbolic Data with Connectionistic Systems

This paper deals with the connection of symbolic and subsymbolic systems. It focuses on connectionistic systems processing symbolic data. We examine the capability of learning symbolic data with various neural architectures which constitute partially dynamic approaches: discrete time partially recurrent neural networks as a simple and well established model for processing sequences, and advanced generalizations like holographic reduced representation, recursive autoassociative memory, and folding networks for processing tree structured data. The methods share the basic dynamics, but they differ in the specific training methods. We consider the following questions: Which are the representational capabilities of the architectures from an algorithmic point of view? Which are the representational capabilities from a statistical point of view? Are the architectures learnable in an appropriate sense? Are they efficiently learnable?